Flagler Data Review

0.1 Introduction
0.2 1. AirSage Data Samples
0.3 2. Review of AirSage Data
0.4 3. AirSage OD Flows
0.5 4. Review of ACS / CTPP JTW Flows
0.6 5. Review of APC Data
0.7 6. Census Data review (2000, 2010, 2015 and 2040)
0.8 5. Review of Census Data
- 0.8.1 Review of Household, Population, Worker and Employment data

0.1 Introduction

This memo is divided into following sections:

Airsage Data Files
Review of Data
AirSage OD Flows
Review of ACS / CTPP JTW Flows
Census socio-economic data (2000, 2010, 2015 and 2040)
Review of APC data for Flagler
Model validation (compare CTPP Trn Flows and AirSage to Model)

knitr::opts_chunk$set(echo = TRUE, fig.width = 9.5)
# User inputs:
projDir         <- "/Users/amarsarvepalli/Desktop/github/Flagler"

# Airsage data files
airsage_data    <- c("airsage/trip_leg_matrix_cusWDH.csv",
                     "airsage/trip_leg_matrix_cusWDDP.csv",
                     "airsage/trip_leg_submatrix_cusWDDP.csv",
                     "airsage/trip_leg_submatrix_cusWDH.csv")
airsage_shapeFile <- "airsage/airsarge_districts/TAZs.shp"
airsage_Districts <- "airsage/airsarge_districts/district_equivalency.csv"

# Define corridor districts
Study_districts <- c(20:40)

# Gtfs network
gtfsShapesFile <- "GTFS/miami-dade-transit_20150626_0146/shapes.txt"
gtfsTripsFile <- "GTFS/miami-dade-transit_20150626_0146/trips.txt"
gtfsRoutesFile <- "GTFS/miami-dade-transit_20150626_0146/routes.txt"
routeName <- "flagler"

# Census ACS and CTPP data files
ctpp_flowFile <- "acs_data/MiamiDade_CTPP_A302103.csv"
acs_shapeFile     <- "acs_data/Tract_2014_Pop_Emp.shp"
acs_airsage_dist <- "acs_data/tract_district_eqiv.csv"

# APC data
apc_dir <- "bus_apc_data/Flagler St BRTStudy APC Data Oct 2015_Xlsx"
apc_tag_districts <- "bus_apc_data/Flagler St BRTStudy APC Data Oct 2015_Xlsx/apc_data_gecoded_districts.csv"
  
# Output files
daily_od_matrix <- "daily_od_trips_by_airsage_districts.csv"
peak_od_matrix <- "peak_od_trips_by_airsage_districts.csv"
daily_od_submatrix <- "daily_od_sub_trips_by_airsage_districts.csv"
peak_od_submatrix <- "peak_od_sub_trips_by_airsage_districts.csv"
jtw_trn_matrix <- "JTW_trn_trips_by_airsage_districts.csv"
apc_trn_boardings <- "apc_trn_boardings.csv"

0.2 1. AirSage Data Samples

0.2.0.1 1.1 Airsage Data File Description

This is airsage data review. The following is the description of the data files

trip_leg_matrix_cusWDH.csv and trip_leg_matrix_cusWDDP.csv: reporting trips between study area zones for average weekday 24 hour period and specific AM and PM day parts respectively.
trip_leg_submatrix_cusWDH.csv and trip_leg_submatrix_cusWDDP.csv: reporting trips between study area zones that passed through the sub matrix analysis zone (in the SZCount column) for average weekday 24 hour period and specific AM and PM day parts respectively.

The following are the list of files received from Airsage along with: 1) number of samples 2) number of trips 3) ratio of trips to samples 4) percent of samples 5) percent of trips

## 
## 
## File                             Samples   Trips          Trips/Samples   % Sample   % Trips
## -------------------------------  --------  ------------  --------------  ---------  --------
## trip_leg_matrix_cusWDDP.csv      9666      2945178.48            30,469         38        19
## trip_leg_matrix_cusWDH.csv       6052      7457084.91           123,217         24        47
## trip_leg_submatrix_cusWDDP.csv   5143      1472307.68            28,627         20         9
## trip_leg_submatrix_cusWDH.csv    4301      4001471.97            93,036         17        25
## Total                            25162     15876043.04           63,095        100       100

Peak data flow shows a total observations of 9,666 whereas total daily flows are only 6,052. Does this mean the daily samples are only for offpeak? 46% of the daily flow are represented from 24% sample?

0.2.0.2 1.2 Contents of “trip_leg_matrix_cusWDDP.csv” and Contents of “trip_leg_matrix_cusWDDP.csv”

This file contains 9,666 observations and 9 fields. Each sample was geocoded with an Origin_Zone and Destination_Zone pairs and Count represents the total flows between these OD pairs. The start and end dates define the data collection period, which is April, 2015. The other fields: Subscriber_Class, Purpose and Time_of_Day describe the trip attributes.

  data_wddp <- read.csv("airsage/trip_leg_matrix_cusWDDP.csv")
  data_wdh <- read.csv("airsage/trip_leg_matrix_cusWDH.csv")
  data_wddp$tod <- "peak"
  data_wdh$tod <- "24H"
  data <- rbind(data_wddp, data_wdh)
  print(head(data))

##   Origin_Zone Destination_Zone Start_Date End_Date Aggregation
## 1          14               11   20150401 20150430          WD
## 2          11               34   20150401 20150430          WD
## 3          24               11   20150401 20150430          WD
## 4          10                2   20150401 20150430          WD
## 5          11               20   20150401 20150430          WD
## 6           2               17   20150401 20150430          WD
##   Subscriber_Class Purpose Time_of_Day   Count  tod
## 1         Resident     HBO     H07:H10 2273.11 peak
## 2         Resident     HBO     H07:H10  283.50 peak
## 3         Resident     NHB     H16:H19  336.66 peak
## 4         Resident     NHB     H16:H19 3715.68 peak
## 5         Resident     HBW     H07:H10 1644.29 peak
## 6          Visitor     NHB     H16:H19 1363.58 peak

Total trips from both files are shown below.

  # Total trips
  total_trips <- data %>% group_by(tod) %>% summarize(sum = sum(Count))
  kable(total_trips,format.args = list(big.mark = ","), digits = 0)

tod	sum
24H	7,457,085
peak	2,945,178

The daily file contains more aggregated data with periods combined and thus the number of samples are fewer compared to AM/PM files. However, the total number of trips reveal that daily includes AM/PM trips although this needs to be confirmed from Airsage.

  # Break down by purpose and period
  samples <- data %>% group_by(tod, Purpose, Time_of_Day) %>% tally
  trips <- data %>% group_by(tod, Purpose, Time_of_Day) %>% summarize(sum = sum(Count))
  kable(left_join(samples,trips, by = c("Purpose" , "Time_of_Day", "tod")),
        format.args = list(big.mark = ","), digits = 0)

tod	Purpose	Time_of_Day	n	sum
24H	HBO	H00:H24	1,701	2,722,739
24H	HBW	H00:H24	1,385	1,267,599
24H	NHB	H00:H24	2,966	3,466,747
peak	HBO	H07:H10	1,332	482,475
peak	HBO	H16:H19	1,429	529,311
peak	HBW	H07:H10	1,137	363,070
peak	HBW	H16:H19	1,068	261,128
peak	NHB	H07:H10	2,286	545,530
peak	NHB	H16:H19	2,414	763,664

0.3 2. Review of AirSage Data

0.3.0.1 2.1 Contents of Daily file (trip_leg_matrix_cusWDH.csv)

0.3.0.2 Trips by purpose and resident type (Subscriber_Class)

There are a total of 7,457,084 OD flows in 24H.
About 83% of these trips (6.2 Million daily flows) are made by resident population and the remaining 17% of trip are made by the visitors.
Most of the visitor trip purpose was Non-home based (NHB) as expected, however there are few handful of trips that are coded as Home-based other (HBO) trips (0.03%). These trips can be recoded into NHB as it makes less sense to assume a “home end” for visitors.
Of 6.2 Million resident trips, 17% are Home-based work (HBW), 29% are NHB and 36% are HBO trips. The work share of trips is in acceptable range for HBW trips (typically ~20%).

## [1] "Total trips = 7457084.91"

Subscriber_Class	trips	percent
Resident	6,201,677	83
Visitor	1,255,408	17

Subscriber_Class	Purpose	trips	percent
Resident	HBO	2,720,440	36
Resident	HBW	1,267,599	17
Resident	NHB	2,213,638	30
Visitor	HBO	2,298	0
Visitor	NHB	1,253,109	17

0.3.0.3 2.2 Contents of Peak period files (trip_leg_matrix_cusWDDP.csv)

0.3.0.4 Trips by purpose, resident type (Subscriber_Class) and time of day

There are a total of 2,945,178 OD flows in the peak two periods.
About 85% of these trips (2.5 Million flows) are made by resident population and the remaining 15% of trip are made by the visitors.
Most of the visitor trip purpose was Non-home based (NHB) as expected, however there are few handful of trips that are coded as Home-based other (HBO) trips (0.02%). These trips can be recoded into NHB as it makes less sense to assume a “home end” for visitors.
Of 2.5 Million resident trips, 21% are Home-based work (HBW), 29% are NHB and 34% are HBO trips. The non-work share of trips seems too high given this file contains data only for the 6 hour peak periods.
Trips by time of day shows almost equal split between AM and PM peak periods with slightly more trips in PM period.

## [1] "Total trips = 2945178.48"

Subscriber_Class	trips	percent
Resident	2,508,516	85
Visitor	436,663	15

Subscriber_Class	Purpose	trips	percent
Resident	HBO	1,011,037	34
Resident	HBW	624,198	21
Resident	NHB	873,281	30
Visitor	HBO	750	0
Visitor	NHB	435,913	15

Time_of_Day	trips	percent
H07:H10	1,391,075	47
H16:H19	1,554,103	53

0.3.0.5 2.3 Airsage districts

The airsage data was provided for about 40 districts as shown in the below interactive map.

# Add airsage district file
shape <- readOGR(airsage_shapeFile, layer = "TAZs", verbose = FALSE)

# Read District equivqlency file
taz_dist <- read.csv(airsage_Districts)

# Append districts to 40 Zones
shape@data <- left_join(shape@data, taz_dist, by = "TAZ")

0.4 3. AirSage OD Flows

0.4.0.1 Study Area Flows

Districts 20 - 40 represent Flagler Corridor study area. The entire study area was defined as one super district “StudyArea” and the rest of the districts as “Outside” super district. The OD matrix generated for this super districts list the External-External (EE), Internal_External (IE), External_Internal (EI) and Internal-Internal (II) trips to the study area. The study area serves about 11% of the daily trips in the Miami-Dade county (sum of IE, EI and II trips).

0.4.0.2 3.1 Daily Study Area Flows

Tables below show the daily OD flows. In order to better understand the travel patterns through the corridor, OD tables are developed to & from the studyarea super district to all other districts, where district 20 is the aggregated study area district.

	Outside	StudyArea	total
Outside	6,079,863	575,832	6,655,695
StudyArea	577,503	223,886	801,390
total	6,657,367	799,718	7,457,085

	Outside	StudyArea	total
Outside	81.53	7.72	89.25
StudyArea	7.74	3.00	10.75
total	89.28	10.72	100.00

	Origins	Origin_Pct	Destination	Dest_Pct
1	7,155.59	0.89	8,255.22	1.03
2	74,591.54	9.31	65,398.40	8.18
3	4,332.24	0.54	3,457.41	0.43
4	6,055.95	0.76	5,162.41	0.65
5	3,464.71	0.43	3,207.18	0.40
6	12,026.33	1.50	11,644.73	1.46
7	13,688.95	1.71	13,006.25	1.63
8	6,120.03	0.76	5,877.74	0.73
9	26,166.67	3.27	25,915.70	3.24
10	50,962.80	6.36	48,879.78	6.11
11	70,945.94	8.85	70,634.60	8.83
12	20,345.92	2.54	21,342.21	2.67
13	31,732.68	3.96	32,990.42	4.13
14	76,594.30	9.56	87,669.36	10.96
15	46,132.33	5.76	44,039.42	5.51
16	41,626.60	5.19	41,924.58	5.24
17	79,137.40	9.88	79,362.06	9.92
18	2,100.69	0.26	2,266.80	0.28
19	4,322.68	0.54	4,797.60	0.60
20	223,886.22	27.94	223,886.22	28.00
total	801,389.57	100.00	799,718.09	100.00

Plot of total daily Origins and Destinations to/from Study Area.

0.4.0.3 3.2 Peak Study Area Flows

The following are the tables showing the peak trips. In order to better understand the travel patterns through the corridor, OD tables are developed to & from the studyarea super district to all other districts, where district 20 is the aggregated study area district.

	Outside	StudyArea	total
Outside	2,394,496	236,047	2,630,544
StudyArea	225,789	88,845	314,635
total	2,620,286	324,893	2,945,178

	Outside	StudyArea	total
Outside	81.30	8.01	89.32
StudyArea	7.67	3.02	10.68
total	88.97	11.03	100.00

## [1] "Percent of Peak Trips"

	Origins	Origin_Pct	Destination	Dest_Pct
1	2,773.64	0.88	3,271.16	1.01
2	24,727.01	7.86	25,681.84	7.90
3	1,530.64	0.49	1,374.62	0.42
4	2,494.50	0.79	1,980.04	0.61
5	1,130.60	0.36	1,024.44	0.32
6	4,675.66	1.49	4,857.67	1.50
7	5,759.00	1.83	5,027.08	1.55
8	2,550.64	0.81	2,533.91	0.78
9	10,519.29	3.34	11,416.13	3.51
10	19,737.23	6.27	21,970.56	6.76
11	27,082.15	8.61	28,923.66	8.90
12	7,409.24	2.35	8,378.04	2.58
13	12,131.86	3.86	13,068.61	4.02
14	33,247.60	10.57	36,088.36	11.11
15	18,975.80	6.03	18,823.92	5.79
16	16,630.44	5.29	16,220.58	4.99
17	32,171.48	10.23	32,553.28	10.02
18	625.63	0.20	730.59	0.22
19	1,617.08	0.51	2,122.96	0.65
20	88,845.32	28.24	88,845.32	27.35
total	314,634.81	100.00	324,892.77	100.00

Plot of total peak Origins and Destinations to/from Study Area.

0.4.0.4 3.3 Corridor Level Flows (Peak and Daily ODs)

Airsage daily and peak origin destinations at Corridor level

0.4.0.5 3.4 Contents of Peak subarea files (trip_leg_submatrix_cusWDDP.csv)

0.4.0.6 Trips by purpose, resident type (Subscriber_Class) and time of day

There are a total of 1,472,307 OD flows in this file which represent EE flows through the region (Not sure I fully flow this file - Need some explanation from Airsage)
About 83% of these trips (1.2 Million flows) are made by resident population and the remaining 17% of trip are made by the visitors.
10% of these trips are External-External through trips (ee). This is computed based on SZCount field in the data.
The share of ee trips by purpose is consistent across all resident purposes and time of day.
Most of the visitor trip purpose was Non-home based (NHB) as expected.

## [1] "Total trips = 1472307.68"

Subscriber_Class	trips	ee	pct_trips	pct_ee	ee_share
Resident	1229302.0	127292.81	83.49491	83.05297	10.35489
Visitor	243005.7	25974.21	16.50509	16.94703	10.68872

Subscriber_Class	Purpose	trips	ee	pct_trips	pct_ee	ee_share
Resident	HBO	379186.23	40185.010	25.7545508	26.2189545	10.597698
Resident	HBW	370170.93	30528.632	25.1422264	19.9185915	8.247172
Resident	NHB	479944.84	56579.171	32.5981347	36.9154243	11.788682
Visitor	HBO	51.36	41.595	0.0034884	0.0271389	80.987150
Visitor	NHB	242954.32	25932.612	16.5015997	16.9198907	10.673863

Time_of_Day	trips	ee	pct_trips	pct_ee	ee_share
H07:H10	674933.9	65511.15	45.8419	42.74315	9.706307
H16:H19	797373.8	87755.87	54.1581	57.25685	11.005612

0.4.0.7 3.4 Contents of Daily subarea files (trip_leg_submatrix_cusWDH.csv)

0.4.0.8 Trips by purpose, and resident type (Subscriber_Class)

There are a total of 4,001,471 OD flows in this file which represent EE flows through the region (Not sure I fully flow this file - Need some explanation from Airsage)
About 80% of these trips (2.9 Million flows) are made by resident population and the remaining 20% of trip are made by the visitors.
About 9% of these trips are External-External through trips (ee). This is computed based on SZCount field in the data.
The share of ee trips by purpose is consistent across all resident purposes and time of day.
Most of the visitor trip purpose was Non-home based (NHB) as expected.

## [1] "Total trips = 4001471.97"

Subscriber_Class	trips	ee	pct_trips	pct_ee	ee_share
Resident	3217870.5	298634.30	80.41717	80.1565	9.280495
Visitor	783601.5	73929.74	19.58283	19.8435	9.434609

Subscriber_Class	Purpose	trips	ee	pct_trips	pct_ee	ee_share
Resident	HBO	1154883.54	105315.88	28.8614677	28.2678597	9.119177
Resident	HBW	816881.18	52825.53	20.4145171	14.1789127	6.466734
Resident	NHB	1246105.76	140492.89	31.1411843	37.7097290	11.274556
Visitor	HBO	150.76	58.13	0.0037676	0.0156027	38.557973
Visitor	NHB	783450.73	73871.61	19.5790633	19.8278959	9.429005

Time_of_Day	trips	ee	pct_trips	pct_ee	ee_share
H00:H24	4001472	372564	100	100	9.310675

0.5 4. Review of ACS / CTPP JTW Flows

The journey to work (JTW) flow data from 5 year American Community Survey (ACS) 2006-2010 for Miami-Dade county was analyzed to understand the transit trip flows in the region. The tract to tract level data was obtained from ACS website and was processed to report out trips by mode and by districts (as defined in the above airsage data section). Also note that JTW is only for work purpose and is in person trip format which doesn’t represent the entire regional trips.

The first table shows ACS CTPP trips by mode. There are about a million work related trips which are almost close to daily HBW trips reported in the airsage data. The breakup shows about 56,000 work related transit trips. The other category includes work from home, non-motorized modes.

MeansofTransportation18	modes	n	trips
Bicycle	other	186	4,742
Bus or trolley bus	transit	1,938	49,171
Car, truck, or van – Drove alone	auto	26,209	778,487
Car, truck, or van – In a 2-person carpool	auto	3,561	74,625
Car, truck, or van – In a 3-person carpool	auto	679	12,818
Car, truck, or van – In a 4-person carpool	auto	192	4,089
Car, truck, or van – In a 5-or-6-person carpool	auto	115	2,304
Car, truck, or van – In a 7-or-more-person carpool	auto	89	1,657
Ferryboat	other	5	55
Motorcycle	other	104	2,247
Other method	other	443	12,322
Railroad	transit	95	1,784
Streetcar or trolley car	transit	33	658
Subway or elevated	transit	299	4,995
Taxicab	auto	54	1,303
Total, means of transportation	total	30,665	1,016,231
Walked	other	734	23,067
Worked at home	other	451	41,519

modes	trips
transit	56,608
other	83,952
auto	875,283
total	1,016,231

Plot transit desire lines

Corridor level CTPP JTW transit origins and destinations

0.6 5. Review of APC Data

The APC data was collected for 22 routes in the study area. The table shows list of routes with observed boardings.

## Warning in left_join_impl(x, y, by$x, by$y, suffix$x, suffix$y): joining
## factor and character vector, coercing into character vector

	ROUTE	boardings_Route	alightings_Route	total_Route	boardings_Flagler	alightings_Flagler	total_Flagler
1	2	2,873.89	2,885.33	5,759.23	666.80	642.18	1,308.98
2	6	453.48	446.65	900.13	464.73	441.90	906.63
3	7	4,222.84	4,242.24	8,465.08	6,022.86	5,690.49	11,713.35
4	11	11,531.54	11,639.18	23,170.72	12,816.23	11,776.51	24,592.74
5	12	3,583.07	3,615.56	7,198.63	1,389.44	1,216.71	2,606.15
6	17	4,463.04	4,458.51	8,921.55	1,015.70	1,156.88	2,172.58
7	21	2,721.56	2,743.74	5,465.30	501.49	543.94	1,045.44
8	22	4,316.87	4,258.15	8,575.02	936.74	925.16	1,861.90
9	27	8,079.56	8,006.27	16,085.82	2,390.66	2,236.28	4,626.94
10	37	4,122.59	4,103.05	8,225.64	2,103.96	1,899.93	4,003.89
11	42	1,500.90	1,459.58	2,960.48	862.34	900.25	1,762.60
12	51	3,338.11	3,347.70	6,685.81	3,550.20	3,650.90	7,201.10
13	57	512.08	534.75	1,046.83	209.41	145.39	354.80
14	71	1,233.24	1,161.83	2,395.08	290.39	289.55	579.95
15	73	2,333.47	2,343.33	4,676.80	321.52	299.03	620.56
16	77	10,023.29	9,767.90	19,791.20	2,360.61	2,028.40	4,389.01
17	87	1,731.93	1,745.88	3,477.81	459.40	463.87	923.27
18	137	2,064.84	1,923.79	3,988.63	377.78	312.31	690.08
19	207	1,765.05	1,689.58	3,454.63	1,765.05	1,689.58	3,454.63
20	208	1,759.76	1,593.40	3,353.16	1,159.94	1,074.24	2,234.17
21	212	111.66	102.69	214.35	156.50	144.75	301.25
22	277	868.09	856.91	1,725.00	237.20	236.13	473.34
total	0	73,610.89	72,926.02	146,536.91	40,058.97	37,764.40	77,823.37

0.7 6. Census Data review (2000, 2010, 2015 and 2040)

Review of households, population
Review of Employment
Review of zero car households
Review of low income

0.8 5. Review of Census Data

0.8.1 Review of Household, Population, Worker and Employment data

The map below displays population, and household and employment densities.

The map below displays employment and worker densities. The worker densities are the household workers which is different the employment data.